{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 输出层的设计\n",
    "神经网络可以用在分类问题和回归问题上，不过需要根据情况改变输出层的激活函数。<br>\n",
    "\n",
    "机器学习的问题大致可以分为**分类问题**和**回归问题**。<br>\n",
    "分类问题是数据属于哪一个类别的问题。比如，区分图像中的人是男性还是女性的问题就是分类问题。<br>\n",
    "而回归问题就是根据某个输入预测一个(连续的)数值得问题。比如，根据一个人的图像预测这个人的体重的问题就是回归问题。<br>\n",
    "**一般而言，回归问题用恒等函数，分类问题用 softmax 函数**。<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 恒等函数\n",
    "恒等函数会将输入按原样输出，对于输入的信息，不加以任何改动地直接输出。"
   ]
  },
  {
   "attachments": {
    "image.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAGeCAYAAAC0MOuWAAAgAElEQVR4Ae3dC9x15Zg/8DfRW5ISKSnGYCrKKJVDotCQokaJUaFSKiE6vqMTCZVDpLNDFCWEooYSNTl00hQ1EjkMknPGjJlm5r/+n+9d6221D8+z1t5r7b3Wfq7789nv8zz73Xute133Wr/7un/X77ruRVm0sEBYICwQFuiUBRZ1qrfR2bBAWCAsEBbIArjjJggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x4IC4QFwgIds0AAd8cGLLobFggLhAUCuOMeCAuEBcICHbNAAHfHBiy6GxYIC4QFArjjHggLhAXCAh2zQAB3xwYsuhsWCAuEBQK44x5YmBb43WXZkk3Xyf72iMuz/7jHArd+/O+zddbZITvrhwvTJHHV3bFAAHd3xip6WpsFfpddtv/a2aLF22fn/rxw0Dsvy/ZfbVG2eMuPZIHdBbvEr62zQAB364YkOtS0Be68bP9stUWLsg2OvyG7q+dkPz93+2zxosXZ9vdB9J4PdezP//3f/81+//vfZ7/4xS+yn/zkJ9nNN9+c/eAHP8h++tOfZr/85S+zO++8s2NXFN0N4I57YGFZ4K6rsqPWXpQtWrxHdtEgvFr6/7tmF/6uW6b5n//5n+xXv/pVdtNNN2VXXHFFduGFF2af+MQnshNOOCE78sgjs4MOOijbf//9sz333DPbe++90++HHHJI9ra3vS075ZRTsvPOOy+7+OKLs29961sJ2IF9tHZaIIC7neMSvWrIAnd71Iuy1d7yzT5vOz/lz8/eOlvEI3//Tflbrfz5//7f/8v++7//O/uP//iPBNgA94Mf/GD22te+Ntt8882zNddcM1tuueWy+9///tkDHvCAbPHixdnyyy+fPfCBD8xWWGGF9Lv3/N+yyy6b3n/sYx+bveAFL8gOPPDA7KyzzspuvPHG7He/+132l7/8JTMxRGuHBQK42zEO0YsKFvjLbd/OPnHCvtnfb7pOts4687ze/o3CkW/K3r/BomzRonWy428ovN3766/OzbZfNIdX3vv5Kf39xz/+MfvCF76Qve51r8vWX3/9bMUVV8xWXnnlbMMNN8xe/vKXZ4ceemjytj/96U9nX/va17JrrrkmATG65NZbb81uuOGG7KqrrsouueSSBNLHHnts9qY3vSl70YtelD3+8Y9PQL7KKqukSeCII47IrrzyyildaZy21wIB3L0W6djfd911V3oIL7300uyTn/xk9r73vS/zkB1wwAHZ61//+uw1r3lNttdee2VvfOMb01LZsphX5mH2IOI9O9PuuiO74pinZSsB1bKv/S9benl3ffMtidtetM7x2Vy4nWW/ys7d/u5z7HrhID5l6SGn8sv111+fvec978n+/u//PnvSk56UbbLJJtkLX/jCNN4oj8997nPZN7/5zeyWW25JHPa///u/Z3juYc099Ic//CH72c9+ln3ve9/Lvv71ryfa5Pjjj89233337LnPfW46z8Ybb5y9+tWvzj72sY9l//Zv/zbscPH+BCwQwD0BI9d5CktWHtO3v/3t7IILLsje//73J1Deaaedsm222SZ5R095ylOyDTbYIHvCE56QPKe/+Zu/yZ74xCemh2/TTTfNttxyy+RV7bzzztmSJUuy008/Pfunf/qn7Lrrrstuv/32OR/yOq+l2rHuUYIkwF6cbbj3GdnX/vX21N9//fYnsiVPW+keMB8eWLzqqNXSZxYfdPm8p77llE3vPt6uF2ZtgG40xY9+9KPsU5/6VAJT4/vXf/3XaSyN4Re/+MXsjjvumPe6qn7ghz/8YfLG0S/unUc96lHZZpttlu6bL3/5y2liqHrM+Pz4FgjgHt+GjR/h//7v/xKXaWn81a9+NXnUW2yxRfaQhzwkcZMPfvCDs/XWWy/beuutE7/J4+aR8ax5Rx/96EezD3zgA9lxxx2XHjheE/B+9KMfnXhOHOcjH/nIbLvttkuf+853vpN4U/wpHrUN7a6rjsrWvge0tzzjln5++q47svN3XXw32A4MPN6SnbLp3V709uf+av5LuvygbLHzLT4omx/m5z/cOJ/gEaM5BBZXW2217EEPelCapK2weNOTajzyd7/73dmTn/zkxJdzCNxnP/7xjyfVhTjPPRYI4O7ArfD9738/O+aYY7KNNtooW2mllbI11lgje97znpcdfvjh2Wc+85nsX/7lXxLlwVv+7W9/m5a9JF5/+tOfsj//+c/p4fa79ygFfv3rXydPyXKX537mmWcmr/2pT31qhtNcddVV0/E/8pGPZL/5zW9aYaHcW160wfuzoSHDH56RbXYPhdJHcdz1lWzf9H+rZUddVeKSlh5rs+yMKYq6jdlRRx2VvGvjsuOOOyZOGsVl9TXJxoHQHwB+xhlnZM9+9rMTD/70pz89qVcm2ZeFfq4A7hbfAddee23yrnnH6667bvaMZzwj23fffbOTTz45PbyWsYB53AbMv/vd7yb5mAAVCsUkwYvnhfPc0TPTaz/Mztjsbm95nTmjivd61X2fqwzEl2X73zMJFGjyiZrgn//5n7M99tgjrYyA5NFHH52CiW1Qd1CaXHTRRdmb3/zmjOf9t3/7t9lhhx2W3XbbbRO10UI9WQB3C0eeN/XZz342BYLwmCRaOGyeMZ1u040O2BKYV7/WWmulh5IGmDIBXTP5di+I9nnSPZ25bP97Ape9aHvLKdmmCYjLetD3nrP3UD2nrP3P//qv/0oxh3/4h39I1IhxEEz+z//8z9rPNe4BrfCsBgVJ/+qv/iqjC7cC5J1Ha84CAdzN2bbykXlSlqEoEIBJc/uKV7wi8dqVD1bDFwAFjvxZz3pW4tJ5Vfhy4D1Z7rs8iHYduGmyqTpy2mq33XbrhPJH7EVwnGbcqlCcJFpzFgjgbs62lY4sAEXS98xnPnMpx0w1gof2f9NqeFSyslNPPTWpVB72sIdlu+yyS/av//qvE+zSDdnx69ztSW82J+F8r4yvi1QJW8t2fMxjHpP06TxZNFYXvFcTuZXafvvtlz30oQ9NgXIJQdGasUAAdzN2rXRUy02BQF6WhBLpyGiJSQef5uq0IKXluuQMwMK7MtFQnjTf7swu3PUeCmQued5dl2X7L777c32USp5UM1/yTX4xVT+ff2/En8D53HPPTXJOMk56fHVEutaAN4+bXJHKieZ8Lg15166vLf0N4J7ySPCoBBt52muvvXYK8NDrtrUJmPG4ydIoHOiHJ8G9/urc7e/RaW+WnXLLoBXIXdktZ2x5t4Rv0fZZv+Lvquyo1e4G9X2/Muj7PRa/4fhsncSJ75pNIgeHjt5kKE2dNn86sYQeG1T4s0idCVDK5qR+2meffVJST4VDxUdLWCCAu4SRmvoIj1oBoMc97nHJi+VxTQIEx70eniCFgxRriheZeo23vPgTMF17j+z8H/zpXi33XX/KfnD+HvfovBdlax911b3/t7Rjd2Vf2fdu4O6jUZZ+5t5flk4Um53RaIlXgPfzn/882dHETYM/Cx6qCoSydeUIoHwmEVS/d/Rm/7cA7imNsRRjGYu5Jhv4CUx1pcnS4xmidkgHVaNrGnDuuuXcbCeV/e6R6a30yLvrlDxypXvfW3unc7OBDnmWZXdeuOvd350XjO+lZpouNEVT/6pXvSpN3pQ79Paz0tAm0vLdI+985ztn5bJacR0B3FMYBnIvqcu4TB7r2WefnXmvaw3ovPWtb01FjZ7znOeklPnGr+F3N2fnL3l2tto9XPa9IP532ZKLb8vmTEm586Jsj/S9rbOzixso9HZ6KVc+z+d6v1fxb5M3iSfaSV0ZdUJmqVlNoNbUUlGt8Etf+tIsXd5UryWAewrm/8pXvpLtsMMO2SMe8YhUGGqSact1X65lvlrP0u55jJbIk2l/yX5/+921Sn79pxKcderUXdk333J3vZK51Cl3XrRH4soX73FRo3VKAJkAnhog3/hGsYrhZCw4qbMoVuUaBbbbkok7qWtv6jwB3E1ZdsBxeSBuXIkVaoOgGqQQd70JRuEzH/7wh2fveMc72l23+ednZ1ujWjY4PrthIN7/PDt7a9TLBtnxgz9Qy3CZrNUeYTMSwC5P3vMZRNxGiVkyQZTgLNzz811z0/8fwN20hQvHB9xShEmllMucJT6TEoYyRlo+7r697a7sqqPWzhYtWpztMWALnLtuOD7bYNGibPGuF2ZNboCjjACaTEbsQgAyCToveclLUmVB9XGijWeBAO7x7Ff625Jorr766lRm1cOqgP2sNTpvwK1+M+WJiaqV7c7Ls4MEOfu87p9n526/uH8T4ZovQp0PoC0uQAa4EJoVhWqGipgpp9A1uWPbxiiAe0IjYnNWdIK62NLIZ7XZQUXQ9cQTT2z1w3nn5Qdla/dsCnz3JsLD63nXMWaKgp1//vmp/gyN80JqsoDViud5K1AVbXQLBHCPbrtK36TRVpbz7W9/+0zXL7arjnrfKsa1WyVxV/anX9+e3f7re/Xgf/m9YOfv51amVBr1/g+LB7z4xS9O+zq6JxZSQwmZ0FGFJvhoo1sggHt025X+prRfacB2D1kIxXfsLG7vQ8HXLqZtlx7YET6oTIDNetXYXmjbf6HO1IxX2sFGICb2prX/IwxRJ74SwD2BYbJriGLzlsZS3Ge92RmczM1u4aHdvXe0VX6UiGIHG5v8LtQmMA+8TeyTqXUze5YO4G5wTHkYgjI023hfNEIXE22qmkjw7eMf/3iqu8GzjHa3BezRaGMKZXJtXLFQm4Jqf/d3f5d03bOkrJrkeAZwN2htShKelcwxBYQWUlObQmF9gagbbph7T/WFYhd1aWyMIc5hs4y2NA5G/ppEn5QE3nXXXbPVV199yjsrTeJqmzlHAHczdk1HVUQKPcLD8rAupGYJLNGI5E1AaqE3ZVupitQzlznbhro0vF31RGzcgMKxofSkmpXYCiuskBybWU4+asqeAdxNWTbL0qa8IugK7Vx22WUNnql9hwZUp512WtJ1v+xlL2tfByfcIxs+W308+tGPbkXA1vZi6Cw8c15+Yfvtt5+YVeykpBqiWjfT3c90Ypdc64kCuGs1570HA1zXXXddKn2qgBDed5rNUljqMUmWZfqPf/zjRndWcf2ClBJN1l9//VZruoeNi63k2KwOj9AeolYfii2xzbSbTEaJMALngqWKdcl8nVS7+OKL073x0pe+dEEoreq2awB33Ra953gq551xxhnZSiutlHYzaeg0pQ4LgPRH5qbsRjUycIx17BA/VwcAlPPYP/Pyyy9v1Y4+c/U7/z8FtKyUxCnGLY70rne9K0ng9txzz1ZJ4ExMNPeAm0RvUu2aa65JZR823HDDBbcarcPGAdx1WHHAMSyN3/CGN6Tdr5VwnWajI7fxAZ6dV7XsssumGsmTSDs+7LDDEjDYimvaq46qY4DqkeknoKjmuEnopJNOGql8rVWXWAduty6P+6c//WnymG3iLCPXT4Fw1fisqDSxBt6+MguoCasfE3je/P80gNs+pkuWLEmcvyJb0apZIIC7mr1Kf1otEkqSfG/G0l9s4IM4RA8H7/FJT3pS8q6UlFUPuulmW7Zcw94mJUWZ6z7uuOOyhzzkIXdvvrBoUaIUeIiW94ccckhaUdkbtEwiDR4ZTfKhD32oNuA2EVoN2EpumWWWSf2kj5ZSn+cLSHCh6tl2221T3W+JYD/84Q+XXv60PG6JOILWdshRjz5aNQsEcFezV+lP4xCB5Gte85rs2muvLf29qh+kTrj11ltTPWdV13haRQoEt11saiJbFvO+JgHcqBmTl4Qc6d5daqocrrvuukuBO9+0If9Jziaghwa55JJL0s73MkUHceJWOptttlmaQOvwuIvjCsBf/vKXJ928vR4Hnd9KYZCyaVrATdEi5f9+97tfqI5GeCgCuEcwWpmvXHDBBWkZiCpQYKru5sF185OWqX2RgwkFh2xF4F18uPPzA9FJAjePdOedd071KdBHXWqAe7311ltq29zG+U+U0/3vf/8EPn6iKsQPeMG//e1v73OpPHWVE9lj0Ljc58Mj/MFReOELX5hWBcXKkzxudueF+0xvmxZwS0Sz0TS7mfiKzkZvH+PvfgsEcPfbpJZ3zjnnnHRT4nabqNch2GjTAvwrb0tW3nnnnZdtvPHGCSDOOuusgRsaTBq41WbZe++90zK9a8AtNoHbzoF6vp+LFy9Oe4g+/vGPT+Pwyle+MtEpgNQ4Udj4vQngdo+ZNPRRUDyPJ0gCQ0WoxjeIqpoWcFt1mMRQJXZQyvtby8O3AA4SwN3QINOp5g9Rr/dVxynxqnvssUfa/swu2nnDcTov7fig9PpJA7csOYCy3HLLZSRguFd0DhD3f7Y68zN/4WNRS/mLpNLvVAhKBtiUOH/Zz5BahSeprnXvy8rj85//fPI2eZz5S7DOxKoYFlDLX/5WchfwGb+99torbeI7H2AP+3/FpAC/sXD9bG8iawK4jb9retzjHpeCoIpZaVZlgoDD6n5PC7j17Vvf+layyz/+4z/GLvBptMr/E8Bd3laVPnnqqaemJbQkhya4ZPI0gT9qEeClUYngUQGJRA8PZW+bNHDbGce2VfoECI899tjsiCOOSPtT2qOy90WJw0OXcVp8+a5ysZQd+YtHi4ZBD0luAZB+5i98vkJX6mIUX1tttVUKFLIV+iJ/+dvEJzaB9rA7uUzHYcBc9X18uImoKeBma9mZ+mUyR5OIKwDGYXtaBnD3PiHd+DuAu4FxsgyUPowDFYDh9dTdPPx5kEtqPc9VFpz6IB5cEq82ALeKeCaXIsgJSPFGlX61I0r+8rc9GE06j3nMY5a+XJOXLFTyPHxx/uLR2ogWDWHnHUku+QtAU1NQdOQvoO73V7ziFdluu+2WgsdWLvnrta99bVohmGwArb4U+17291zl4TpNAr6Hg+ZlNgXc7gNBUuc0yaHPrDpQEnfcccfAW3BawG1SoXJClciezFUwAzsZb/ZZIIC7zyT1vGFPQQBl+Vr3noLFB58eVvabwvQoBMAGJNoC3KSIb3nLW1KfrBAoXwCJvqI60B9+5i+eoc8MeqFMUA281vzlbzp1WZq9LxX41OLIaZjiTwFjahzSuOKL16rPgM7k++QnP3mp1K4saFOb7LjjjmkidZ3oH947Hbe6IMXxq+duu/co+k0eaPytHKxa5pIrAvtp6LjReHj3CE7eO3ZVfgvgrmKtCp+lSOB1nXnmmY14EzwW3ixdLqrAeXj2z372s1sF3MDxgAMOSJ6V37vU8j00TcDzgTZgRkNJtKFPBtDFbEurAJQMxUm+UmrCFrJk9duKxYrvwAMP7FvxFScOfdF312fVUmzFzxXfr+N3zgynxjNCqhitmgUCuKvZq/SnqTrclDxvJU7rbkCBjMo5LIvz5uHLPW7eVG8rctyTyJzk9e63337Zgx/84OQR9/anzX9TldBfs3EvcCtlwLOmIAHK6CA0yLCNAXjgEnAEPpsEbsemHkHLyKYkuRt0H/iMQlOCmGussUa6PlSUa7CCoVpqsknAQe3xuCMBp7qlA7ir26zUNygXll9++STZk5pcd/Nw4QcBysEHH7z08OpNtAm4UR52PLFtG9qiS43nCmx7QZvsT7CT1NM4WPZbAc0FyIKGVkMCs3N9ri77mEgEeSl49K23Wam5rt5JKf9bELnJRlUkjiCD1+QSrZoFArir2av0p8mvLFdJ4ZrY7cSNn6evU2JotLyCdh5ImZGD9OOCdf4fkBZrQje1LBYc421SbHSNKiEffNrTnpbsteaaaya1ingC+aH9EvHJqIkyTfq8SVVAdBLArfLf2972tqF8uklUkpj7VJAQtZPLKpVHaDrLlZ7dSlFMRswjWjULBHBXs1fpTwu2WWZLjhGAq7vRhgMRS02g6HweNoEpEjYgkQOlQB4tryzOfFnMswL43qd1bqrRRVN72FShiZVHU/12XOBsOa9sgd1rANuotNfnPve57HnPe17y4IfRKXVdi36To37mM58ZeMiyk3TZzw08yTxvCkxagXg+rFqiVbNAAHc1e5X+NP4QiALvJqqf8doEJ3GZOFZ6cdlnvCgaXoqIHCgFLu02Qmong08aN27WMhXdIqDWVONpUmZ0NcmCnesAWioWMkQlbgdlMI5qf/0z7iZtCiMgaOytFvLxH/XYTX5PsS22kP3b5n42aYNxjh3APY715vguUAWgPFw3aRONR4RfzTlWf+MzpTlbwucek/fodQWpel/e9/mmmsCkB7QpPXtT/W7iuG9+85uTZp23WVdtDsfBtSvXigKzKQItOmlkW5v7EtfPmeDUDCqK1da+t6VfAdwNjYSbUdBFMsSkAlINXcpIh/VwUg7g1Hn59NGT4HZH6uyEvkRhZLXjfphLW12lO+IUaBHJS4BbTIW2vY5VQpV+VPmsmI/V6CMf+cjwtqsYrvDZAO6CMer+VWIHOuJVr3pVSgKp+/htPh6Q5llSZeDbywbx2nxN4/aN9A5dgj5DpdXRrJYcSyBS2dZiIa98xVXHeeo8hhwHcY/tttvuPgHyOs8x68cK4G5whNEQz3/+8zOp1zjohdQANV4bcEvAiZZltkIjs7MKk4izUBtHhlrnlFNOGVgIbaHapcp1B3BXsVbFz+Ke8Y9uUnsNLqSGS1ewid65ieBsV20pBV7yjjIATUvu2mYjqzAUkdotEsVozBc6fTbqGAVwj2q5Et9zU4r0owrUyaarXgg3qglLBt5qq62WamVMIkOzxHC04iPqoNj6zD2x0FZhSjLYD1OuQTFprBUD07FOBHBPYMAoK6QTS3euS00wgW6PfAqKGg8mpQM+M9q9FhBMRJMo7KRUrUluoTRODNCm3VbFMNroFgjgHt12pb8p+UKJUF6W5eGsN4kqdOISfOoKws2SzdBItPeqBVIeLYRVGG/bRhUrrrhi0pmL/0Qb3QIB3KPbrvQ3VUKTiKLGhSxFD+6sNskl6it7QO2H2VZlw7TtLykKeAteLwQQk1ovSK/Imd2Moo1ngQDu8exX+tuCUlJ8ed5+n9WGGqEksUtNXVrlWbQVnf/hhx+eKBMgXnfN9jbZjJ5fkS2JWFYYC4EubNr+AdxNW/ie40tL9oDaUYVMrontzCZ0KUNPg8PcaaedkppE7ZQ2J4EMvYgJ/scnP/nJFPug67Yf56AqfhPsTiOnck0CkmqR21pulieoRgw45KAB3EMM08Tb1BVkUB7U0047baZoBA+klG6b1doWLFq/BUxk7KRQFe4foOU7FlmhqJg3S831KuuriqV7XhXChcDnT2IMA7gnYeV7ziEpBe+bb0gr9XcWvCxLX9y9pbC0a3TQQgjCVrl18Ni2ZTvqqKMSiCn4JeaR765jtxoTX7HUbpXjt/GzVmAAm25b4ato9VkggLs+W5Y6khRlO9eomKeyn5u7y03wUUVCnqNkGxv3rrvuumllQU2zEAJvw8ZP1TulVQ855JA0mZmw1W1Zbrnl0gRnkhPEVV/E/5GM2mpsFjhg9VJkSD7kIQ9J2aKjlsMdZtuF/n4A9xTuADVM7P7x0Ic+NOmdu7YzTNFk1ALqbygYhMMHVKrT8b6pJhRWWihlO0nejCWbCNJK9VeDe9VVV10K1MC6+JJV6zt2TKK4sJs9mwnodbWpB26TYpt1qGWe14Xv6vW0sd8B3FMaFTuOAzybHkhWkVXZJekc/tID+qIXvSjtvcirzANP9itU8AgdYBcgwVgywVkrNIXmQm0IPKt+CHxf97rXZU94whPuA85FoC7+LglHcai8KQ2wySabpEnw5JNPzmyW0aXGHpREdrYxkW+99dZpl6AuXUNX+hrAPcWRkmHo5pYabqf2QVuNTbF7Q0/tAaXFtbxX/RBY9S7vyd1sW0a7q6gSGmXW9LtoLt4xtQTvUg0S1Ee+F2gRpAf9vsceeyTQzydsNNrll1+eaDST3pve9KZOqY+sHF7wghekjaEFW20eMQsxnKEPwhT/I4B7isZ3alF3KgzUgqAeGYC7vhcAACAASURBVF3+IE+5awNPD5Dtyk3eha/lTReXwsW+47ftZQjYAbwt1lAIRSlk8fMDT9jCNykjBBqlrNvb0w72g4B5rvdkTYoB9DbJWV/+8pcTxUI6ass352pz02dZkUq1ckJMOLbLi9acBQK4m7Nt6SPbk/KVr3xlohxsrEvf28ZgDg9KtUOp+zxMe1iWqXCnRjTFBNpE8NLOQKiirjbATbqXb7w8F0D3/p+9Pu1SgwpBsQxrViuoNJy3TQfsIFTnlmfDzlvlfZPuzTffnL33ve9NSVfuCdx21wPuVWwwrc8GcE/L8j3nFcBz0/PeqAvs0g28m9xWrKcLA//0cPKoPIxqjwi02fGcBrmKYgTY2XjXDjCUBgpvWVp30ePODSUYa6xs2NwL0MP+RqegjcqAm9WK4KZjrbPOOml/RjJLtNQ07ZZz+6ivfffdNwXZ6fftHznL5RzycW/DzwDuNozCPX2QoGNXdDQEyRhqwaavvfzxJLvMy+NRCaThqvOA2ih6Yw88MBK8omF2ncN2Ip/kNY56rpwisHcicOVNDwPs/H2rDius+aoCCnaaKPPvmdDREE984hOzk046aapBP0FpKyj0nonIeEqumbXg86j3xSS+F8A9CStXOMfvf//7JCdT24F3ShON51S7eZISMd4wr593+NjHPjYFIo8++ujEY6JKKCBGWQ34zrXXXpvqdODIeayUFXfcccdSK03Tm1zaiZK/WBUpIIbDz0F22E8rFZrt+SY9tt15550TUDsWb9bOOfT/qCZ244kbj0llW3IqbEXnvqR80Sd9EJy1s/x8E1FJc8bHSloggLukoSb9MdQJQBDE8pDIQJOcgesU+KmbA+ctKfL/zW9+M9UNp3iQ8aY8q6CpiYPcD1jwugE6PXqVVgRkkkFZhGp2AyIFlwD6KJNBlT7U/VkUkL0kBRKHAXbuiYtfuMZhjU2Mr8SsXJnCozVxo0gA/oc//OE0HqgT3rfaMOIOOHE7u//mN7+phUYh97TV2tVXX51kjsaHAsq96L6wo5PEq+KYFn8fdo3xfj0WCOCux46NHcWSmbclMxF9QsUgaGW5fMMNN2QedokfHmpeD+BDSQCU4oPkd+8BaA+lZT4KxnfJECkXeL5qqUgMImsj7aIgAQZ5w83ipwHLhz70obG1xp/+9Kezpz/96enaZNqRw+lnVxragBcKuI1PDrhFEJfWziMXkBzUXK+VlhhA7wSAHjH+QDRvxhnF5D4QL3BeE6BkF8BucrA6M9EaX+MsHmHMfdf4e/nde/7PZ3xeP2ixUVpWXHbrsVLA4wuUUgjx8kPml4/GdH4GcE/H7qXP6gHxYPHAycfwnjwevCpvDNXAI5PwctZZZ6UHTqTfwyeBw8Pp+6gIHjWwJzcjyyPlk62Hd3W8VVZZJXn4Rx55ZPa1r30tfcf3ixOAjtu9ROEgr3E3vQVawO/1r399Cnyih0444YS+c5Y22AQ/CDythNjOuMiUNAnlHnYRvClphpW5tXIhCRX47QX+1VdfPVEUvXEO40KVgtJSsAy1glbTFy/0Fnne7rvvnlY2JnqrJhOl8UfH8O4/+tGPpglDJq8+iKusscYaaTJYeeWVEzVjopZcxOtHmQSXPcGbbMipAriHGKaNb9NQe1AtixXt4f1YfntA88JVwNT+lgAFF61QP1pDLXAPpa2jgL1aKYKD/g/wo2F40HavIfGbi7IAGIBfwGy33XbLyP3GbeSBwA1/yrMT/CqjvBj3vKN83/VTUJhAASTQU+3P5MouJsAcvJdffvk0NoO4aGCsONd2222XvlME+vx3K6z5JJdWXRQejgWg6ahtD4basIIymWy00UZpouWZA3gTpN9dg/vFvYGzlglLfkjqyXtXbtY9596L1h4LBHC3Zywq94QHh1/1sAI6ACxT0YOKWgHO0q9xoQAbKHpAcaiUADxuHpeNfaumV+PYTRiObcebUemNojfPm+M9mlB4fbxw1IkVQxua1Q8e2QQDsNn2iCOOuE+8AaWhVgutNvCVAema8nIA+XX43Ec+8pE0ceYgnYN9/refxqzX286PMewnIBerOOecc5IiSEkFnDSPGr1iMqdBR4PIcCRDxWG7j6iYBBvnC6AOO3e8PxkLBHBPxs4TPYvMRMtaXqzgEs5T8owU+6ogMKzjlsuWz7nXpkRtXUto/eX98VZNRLy+aQOJCYYtgRwpo4kR1TCo4ZdNnoDXRFmceExwJsm3vOUtfXx2EbD9LtZggp32tQ+6xnhvuhYI4J6u/Rs5O5DhHQJSlEcxYFn3CVEsCgpRNxSDmOOcR9/px5csWZL4d943Dp8nOa0m2QbdgAYRROWVDpuogDPqRO11FEZxVeF7PF6eeF6Luxew879RWgKE+OxoYYGiBQK4i9aI3ytbgBdquY0asDSvkwsVTKUTVh4W1yvQppbLJJtVCiqIBBIfbDIR4J2vAWvAnlNIqBIKHTGFnEbJAXrYT3EJgcRhE8R8fYj/n10LBHDP7thO7MqoK3jdAlvUKHU2NAPeVaDVOQRj1bvGhzfZeP28Y3EANTgEfd/97nePlARlpSBoKXlmGEgPel/VQXGMHPybvN44drcsEMDdrfFqZW95pQCOJ0lhUafXnV8wZQXJG4kaRQQwrxpQzY8130/aZjy7SUKaP9XFqCndJh5KHcHMQeA813uCnNHCAoMsEMA9yCrxXmULCIZSseBl0Rt1N14nDt2xBURxzZQ0dVcZNOmoF4MaoaGW1CKwOyrPjDIhE9x8880rAbdJEEUTLSwwyAIB3IOsEu9VtgBglRlIHkhDLqmkiSU+8Fb7GVcsVR6dMG4SUH6xEpesHOicySel5Htv3Ab0eeyOp4RB7mUPkv/5P+9TrQiIRgsLDLJAAPcgq8R7I1lAujT5Gi5a9mVTVIbOyd5UQwX/LIhHmjiq6oTqBjWiRKlaHDz6JrYOE6AkIdRvaeQ5gPf+BNw01zTs0cICgywQwD3IKvHeyBaQSk2DjWZoWgEiiUXiiFRxJUbRKFL7i/K7+S4En00ZI/iJz5ZUpG5LE6uFvC8mGIFc9T+8BskCZS6in6KFBQZZIIB7kFXivZEtoHiRbEwcLZkgT7bJxquX6o1aAOBSvctumyVR6ZRTTkmBQ7pqNIk0+7nS/ce9FsFKNWcUk6JU2XvvvVOiUa/XjWevMgGN26/4frcsEMDdrfHqRG8F82iuVcRTBhaYN9lkFn7xi19M6dyoDvU5aKbnUrdIkFEbW9o6akSiS9UytaNck4nBikQQV70TwdVTTz01eeAUMzmAD9qPcpTzxXdm0wIB3LM5rlO9KkCNbuBVoiAUxZpEs/EyFQiO/WlPe1rypnsr8qFG9E29ccWseOq87qYnF9dvdYDOUYbVpFHcZJk2XZ8EdyX6qB8TLSwwzAIB3MMsE++PZQEc8UEHHZTA8cUvfvHE6m0AxzPOOCN5+w972MNSpTwFsWQfoh7s4iKYqXyqNP2mefiiET/1qU+licImCIpA9TZ9V0VQ4o862tHCAsMsEMA9zDLx/tgW4O2qQIc/5mn2Vsgb+wRDDoC7thGAsqZ4b0BNMnjIIYekHX3UPkGNKEcrQ3ISjXeteqOND8QAKHB6m4mF549jD3671zrxd9ECAdxFa8TvtVtAkE2tcJwu9cakGo8fdWKDA5SIUrHqnajWp5yqXX8m2ahE8Okmk7q3nZvkdcS52mGBAO52jMPM9kKJU/WrURNqV09a4mbXF4CdB/1oqAVPJ9VQNN/5zncSaEudtwNNtLDAuBYI4B7XgvH9eS2gVreMQcFKgcBJNB63DMucz3Z+QUu898te9rIk+2tSq51fowqHkpLo2nnd0cICdVgggLsOK8Yx5rQA7bLypMDLBsRUHU02QT51TGQnoidU9SP1o9Sw3RvZXb4RQpMlU123yUNiD3mk2tzRwgJ1WCCAuw4rxjHmtQBNNc9T8SaSPbK8JppJQVU9oC0Lkodf5LMpNuw+Y4Nk4G1n+x/96EdNdCWl5eO0efqShCYVCG3kYuKgrbJAAHerhmO2O0N6R9etvgjdcp3gTbHCq1f6lXLD5guScgbRISYRiUEmEeVWqU3w0HVqufXHnpm2H7PnYwQkZ/venvTVBXBP2uIL/Hzvfe97k8oDsNaxOzxz0jyjJNAiK620Ukq17028GWR2iUGom2WXXTZlLqrgV0cj5VOkSkam7c7UVIkWFqjTAgHcdVozjjWvBdASaoJQmQBxRaHGaRQib3rTm5JeG/XhmLzbMjpoemnUCbUL/l1izDvf+c6Ra2+7DudVf1tfZEBKZw+KZJwRju8OskAA9yCrxHuNWuCKK65IoCYtfdhO6WU6kFMj1Crqc9NnjzIRmEwAthrcj3/847MDDjggu/HGG8t0oe8zKv9Rj1CvKCBV1wbKfSeKNxa0BQK4F/TwT+fiZTZ+4AMfSF7yLrvsktn6rEr73e9+l/hsYC0rc6uttkq1qwfx2WWP67tS5e0LiZfGT8u+VMCqSpNSnwdGL7744ipfjc+GBUpbIIC7tKnig3VaQIAQ4NrFBnVSFnRJ/WRjSqoRhESTlKVGyvQfdaI0rDrZsi0/+clPlu6bOiOUM8svv3ymLsmo252V6Wd8ZmFbIIB7YY//VK/epgvAkbKDxnm+Otg33HBDoh/w0WgN1Ejdag0TCM23kqsq9ZHyoU7KnAdFwttW5a8pieFUByxO3hoLBHC3ZigWXkfIAYEdcJSgIstwUJMkA+TtL4mDVs/6nHPOabRo1S9+8YvspJNOSrJC3v1ee+2V4eaLQc/i7+pn4+ztV6lGSrSwQJMWCOBu0rpx7HktAOTsr2jHHLREL72gip7kFSnrPO2XvOQl2de+9rV5j1vHB3jf9Oaki2gZqfJ4697KfgKSL3zhC9Okcuihh6ZT//GPf2x0Yqnj+uIY3bVAAHd3x25men7ppZcmFQZNNaDMG/A7/vjjk0cOOI899tisjD47/35dP2Ve8riViEXrkPgJsGoCpR/60IdSGr1Aq1WD/zvhhBOy8847L+pq1zUIcZz7WCCA+z7miD+mYQEVBGmpV1tttWy//fZLO8UoTKV+NS+bJtou7j43rebcqJNnPvOZyfsG5N/73vdSxiUuXOlamZomGxtISKlXDfDEE0+cVpeXnhclhfqheddnqxx7gd50002Jzx9FQrn04PHLVCwQwD0Vszd7UtwrGZu0a8t4SgzZhXWmmNd5BSgJoKKSH55YDe0dd9wxs3/kNttsk33mM5+pNR191L7zrmVobrvttsnztoMOFckqq6ySpIQCkiYYKf15GVl8vPeLfPio55/ve1L22fGrX/1qduaZZ2bvete7Urq9YKm+2okIpWNPTv1iW2VuTZD77LNPquEigUn8QM2XyPicz+LT+/8A7unZfuwzqz4HmNW45j2ptyFtG3jYZeW0005LmYSW7TTKalPbCUaQzWa5NqqlocbZ1lmnY9QLww+vtdZaCfTI8V75ylemrbxGPV5T3yMZBHqLFy9OfbUpMg9WQhBteQ7aftKEv/3tb29ktWAyQMsYf160SWXJkiVJzkitw+t3fqsWdrWhBPmlDFFBXr97Ly93SxPv/wC7CUkddfEF1wvE59p8uSlbx3EHWyCAe7BdWv8u749npWSpwkpqYqAaAF4ROAb9DnA8yJtvvnnytE4//fS0O42JYFrN6gBNgkeWDo/vNrG0sVkhAGlBS/YVWFWoSp8H2Rs/j8evu5l0TdCCpmSIeV/QNnvuuWe6N+jJ7W9p6zR0j0la/62+3EMolJtvvnmpl674Fh07dYx76X73u1+SRSqHS9kT4F33KI52vADu0ew2lW/ddtttKRAGqPGq9k5EJ1BcWO4CPmVK7e8oScWS18PGa+KN0T2/733vSxI8pU932GGHlCloac9rxMmiKXjlk0zVtqXZdtttl/rwpCc9KVEmALytmYdqj1gNqEWi2qG+r7feetmKK644ELgBoMBlXTJBqhqZnWzFm95kk02SnBIHrwKjdH1eOO7ahEipA6yHNddj0ua9A3c6disy9lcKAG0FyN0jPHnvtXVSHXaNs/Z+AHfLR9SDd+WVVyZ1hY13BccArOWsutY8JBQI79vDRnUhQOZhLRY3ooX2cPKyTAASXqRn87ZlLpoM7McoEAiM6KrRK4JZTW02AChIAHPQBoZqlzgv3thyvW2bD4gdmAxNmmSMbG8CzEGbh9rrdS+zzDIpoGlS7ZU7lr39UFk2gnjrW9+a6oybuN0L++67b7KXScG4190oaqwWbK686667psQnEwYw5yAEgNdt8XLHC+AuZ6eJf8pSludjKQywecU4SBwqzhSY1xnwErxU5vSNb3xjol1wo8qk8uwAlQe4OBGMaxDHA9BPf/rTEz0CpAXWNFmKgmmW/zY9mMtbHLcfVb9/3XXXpQCflQ4Pl/fL25bm3gvYvX+jpihPqoybz7LH+eefnyYKdBhuWsCRdHKStkGzCHoKcPL0ceh2GnIvBoVS9U4a7/MB3OPZr5FvA+2vfOUrCaSXW265FDxSac4D8uc//zmds8rDX7aT+THRJICCCoEnibbg3eNC6wAK3KwHHjcsSEbJUFS8SH2X3i5QZnVxwQUX1HLesnYY9jnAJabAqxa4o+c2wQzysntB2988b7y4VVTZZgWFX8aTGwcyRIHo+coDlD1+1c+5R/JMVvcHGsgYKho2zRhJ1evo+ucDuFs2giL4HlQetoj/gQcemDwrD/Coy+xRLpEHLqCFykAJoAYszwGXgNaoDT1DkYEKEVSTKo4y6W3A4fDDD0/cqlXGoM/0fqfpv3mbqCQrEXYxmZpYB4H0sPcEhU1UJJpzNROkyRMlBrDZCo1k0qtz5TNXH+b6P5QRrT3lksC41ZH7xIQbrXkLBHA3b+NSZ/AgCAriS3maltV4xVtuuaXU95v6EM+OppdUb4MNNkgBOUFQfGuVxltF+/A41f7AZyubOpfn6Npx74ALFz9N8EZb0Tyr/U1eedRRR6VxGgbQw94H9Ow4V9o+2sH1ijU4nwni8ssvbyzWUGUcez+rr7m2HXWC1rvkkksm6mT09mkh/B3A3YJRRn/wPHO+l3flQW1bO+uss1KCjE0CPKBlKvpZWksDp2ixxyNlgmQPHHeZRs4mGPbkJz95ajYxqcrsRAkodGUSwr3zNHG9aAw0yDCwHvS+7MpBkzJPnDdv2zMTHF12E0HHMrav8hnc/2677ZboL/QR/n2uSbnKseOz/RYI4O63ycTfkRYNFAQf7Uo+zdTu+S4ev4qffuADH5hUDfNJ9qhSBDxpx3mavG4PdM6nz3c+fLsgIGAEdjzfSTa0BLUO6soqiLRS03+2QBXQPRu7HKDtYTkXkOPEV1555ZTZWLwWKwqTIxqJ1FANlC418kMqJxOZDFi0WNlx7tJ1tqGvAdxTHAU8JokY0PagKkrU9roRHkTqDzytjDurBJK4Qc314Gh5peRjAq6jqA8ERaVo89hx7JNseH4yOGoONbqLXiQenofMHqgjoGtSA/CSiHIgH/ZTwk6emGOCMIG7Rrr8T3/600sD0ZO83nHPxek47rjj0mrEPY0SakNW7rjX1bbvB3BPaUQsf3P6QFIDaVxTeukmLtEEgz4ANPTfdOS5d0W+hqOVwef/X/WqV6U0+1H7QXEijV+w1kQgYWcSDUXCo+Yd4+SVFJirASheuNoqPE8SR5y+iXmQB86zNimwpXuBLpu00IRXVNnMdc42/p8yDLlDQnkiczNavRYI4K7XnqWOxtMWiJSNhr8V3Oli431K3gGolB8y9qhfSMME1XjkwL2uYkU4VBpmyUGT4H1zBQxvm+eYT0xlx0ofrTKOPPLIBOBKwlLnoBJyCaHf0WO8U2oVSS15q3q+/Htt+Ok+AN4rrLBC2kHo+9//fhu6NTN9COCewlDihbfeeuvEi9Iod1n/KhMTbUKTLSmEhwmM8NkSekahRoYNiexAHpzkIMEvHnETDWDyeE0QgFaAcJQSAI6DAjHBkXKifKysyObQRzmFQguNQ6cimqVmwlYzxSR+8MEHz9KlTf1aArgnPAQUFlQjvC+eFuDresMD55X9eNk8YzUz8mShOq9P0gvbPfe5z01BwzqPXTzW+9///qV8szGrw/u10sKJ8z4pcmwSQS0DwMkLyyptiv1s++/4f5O5RKoPf/jDbe9uZ/oXwD3BocKBog7IyHCb09Ql133ZvElBNZ4jD7WpRlXCe5NirtjRKJ7wfH1Tf8NmxF4nn3zyfB8f+f8l1KDK1GqZ5cQVQVuxCbEQCUTRxrdAAPf4Nix1BN6WTDM1LuybOIsBG1y9h9NGA7TOTTUgJ+hJcqYKYp0NN6sWDOpH1mpTKyLZpyo0muisTiaZFVunvcoeS4kAtBOJYxOTbdl+zMrnArgnNJKWjDIOcb9Sp2e1ATtBNuVjm1pRCPqhmSTz8FaBYB1UhjGh08Y/k+rZlKKpJnBHNgi8Zx202fCqq65KFJqyCU3atanxattxA7gnNCKy4SRp8Dx++tOfTuiskz+NQljkf9L2qUyaajxhKejSrG1iYEUzbqPFFn9YaaWVkqSvieCnCcZEg/N9/vOfn8k4XAiN1FVAmQRyVvn8SY5jAPcErC17kLfNQ2xbfem6L98DakXx4Ac/OPHDNL1NNdpuXis5ItAdRwcv/nDAAQckuaH6KE0lQqkMSONuYpMJupCa5JytttoqrWbw+9FGt0AA9+i2K/1NcjnJFYoFzVcVrvRBW/xBnjZdN66bFrqpxss+9thjUxEqth21qD+5Hm9QMBKVRXZYzJCss//qkwjiinOozb2QGpXR2WefneqwKE8bbXQLBHCPbrt5v2lZ7GZVmEhGHK9wIaT/CkxKg8cV27GlyYZqoLdWN1z24SiUCdqF6kHgmOqnqaZvNqWg25YWvhAm8V5b0vVzYmQLmyDbUKK2t49d+DuAu8FR4rVJQlHPw64hTXlxDV7CyIeWfJGX+WyS6/bgy2pEQ9F2S2iq0vIiVlQkvMAmi1gJUKsqqLqizYYXarOvKeBWFmAhBGabGOcA7iases8xZd/ZFIFErmnPs8HLGOnQvEulX9WULqZxj3Sweb5k0wdqFpsUCIxWkSKiSDbddNOkrc8r/81zupH/2/FxvDaSGJXWGfnkLfqiUr0yh90bkyhd0KJLr60rAdy1mbL/QJbC6iqjShRhmmbj7aMEKFpolUn10DZ1yeh6r40nrECT4KEd6JtsJgnZjZbggn60wmWKNLGFgKRKfjIlFcdqsqFH1NgmlWz6XGWug7drxcF2sjYFTiexKnQ+5RGURhhUk7xM3xf6ZwK4G7oDAJcqdtQVlobT5DOpLShbeKV4ZynjuFybNdRZS6RoSmCqjrV0Z9mBTWm6i+ek7ZaVarIEDvM1iTZs8exnP7vxjD4TiQ0k0CQSbspMLPP1f5z/B9p2NrKXKJpok002Sbx7mc0xxjlv/l1Zr2q6n3POORO5N/LzzsrPAO6GRpInozYD3SpVyTTbj370o6Sa8FCq84y6wLWiCGxsUGXz2irXYfLaZZddEoVxxRVXNA5WJkc1UwCR6xuWXm1Skbnq+lVoFDDs9TTrXonYS/RFL3pRCoA2lY1ZdmxcmyxeW4xJjJGSrhb461//+sQ9o4+aLnxGXaLcLQ1+k3GFsjbp2ucCuBsaMUvA/fffPwEDTm9ajefJ8/eg5o0Hbqs0xY14p5KDmmrKvvJqURFVuOdR+2MVgUcWGOXNDWoAnhJFv3jBvaCdfwd9YKMDunQAN44mXTkAyhV9q3tSyPvb+3PYeUzkFB1FwJRsxMFwTwjyNp3daNKQnara46znNvSOSx1/B3DXYcUBx/CgU5LQMue7nAz42Nhv4akBEb6Whwkci5JDy+FBqg6gZDXgQRXQa6op0kRVQ2c9zu7wZfuH+rGRgWW4zQ96MxNRFHkGn8xLtMWwRqGCN2cjn1WS1fGspgREhwH+oONJcReME6y2EqmjmYAF9yh4xC38ZGPv5clIwJskVUKR/zeR048PopLYwrV60cc32WxK4Z6QAt/k89HkNUzz2AHcDVmfx6I4viBMUx5FzlPa89DDJsjGkyxuF+UBHgQUQB6lQF/eZO0U1AwVhddtt93WkLXve1iBP9mPVCb2uyw2WnrboPk/y/W5GqoLsLCtDQFoxcUIgLm07Sq6fCAl8UbJA1RNHY3nfPjhh6f65Hn/lFU47LDDlnrTJnFUSD4BCQi6NwfdEwLorlU/m5Yr8vbFWewuZFORaNUsEMBdzV6lP403VfDfQ0S/W3fDk9pEV1CJN6hOtf0Yn/KUpySQJD0b9HDm/dAniSD5bu35+3X/NIkAUZTMpHZBcd3oIbYAvLnMT4DUeJiwAN4gr7N4/Sa0vF42QMtfyy23XKJZaJHZDwDxzueqp43fBp4Sk+oCbp40p8BkAJD1T0amCSUPfvK4pZrbRk3g1n1ixTCo8cQdg9KmabUH2klf3INNOg6DrnMW3gvgbmgU1WJwU+INm6AILI151zyWYrYfIPfwActhyQ0eZg+3TQ9E95sMllkSq4Bn+y81uyfZJHiYMFBW7CUQS1NvJWRvyPkajlwAMwfsYT95uXTJ6pArM2uy4lHm4Ok8zgvolQCoA7iL/DV+WmDRtm6kl4NiCWgecYZhzWTneq0o/EQFNdlQWlZjtnDTr7mcjCb70dVjB3A3NHJAwgaxCgoN83DGOTXO0g4qdhXH2Wo4V7ppACM4N0wZYCLh5QD3r3/96+N0Y97vSjRBV9j4IN/k1/K9qZfJyrGBo7rdPE00Ak05CoBd8Ld54kf++d7+uDA2UsVvGGAPeh8QmTypW4yLceLpm0B43NQ1RdCd14AlPyCmwtvmLBjTHAj9pGhBR8xVA56HfcIJJ6QNPiaxMmJvk5j+vuMd72hMllrSfJ37WAB3Q0NGU+whxqPmIFHnqQATTwvw8Ozwnc7J6wIopFbDgBt/ueOOO6aHuUlv2/XitYGYHVlbygAAIABJREFUScwmwoDL+SfxwuUKCqpBwpME4FQd559/fgJVG/kO6wevmSJGDGAQQM/13uLFixMdg74wDvh99ArPH7XRBHBzDthZvyT4oEc0AMnbts/pXA6EiU0wl82aKGfbe++zAQWQuIwV4zTzHHr71oW/A7gbGCWgCqSWXXbZDGVS97Kz+OCjAATRcLY8vJyTtUPMIODG6+afn0T2niCodH+Aop6IfgHSSbyALg8bp50DrQQY3i9AnasP/l+52FVXXXXpd/NjjPpTBi3qqDh+dd5+OGq8vuxRck+N04Czzldlg86HwrJRsRgJ3nxSjSTRhGaCnMS9OKnrmsR5ArgbsLIHU5ILj5tGuokEF+cQ4JFoovg/Lp33LAsQsAwCbkBOT8wz75XJNWCGdEgTBY9Kn/C8Aqm5yqTpn6gDQGYC5W3rA66dfti5STUH9cH79MW2RjPZjArUzku9IijoGI577bXXNgbcJvEjjzwynctEzmFAe4hjOO+ghg93P+DmJx2DIFUF3O7hpuqfD7rmWXgvgLuhUbQ8RQ/gugcFi8Y9rWOKyuOObTycN0keQKIXuFEqvD3L6GIdaBOA/xMsasITpF5505velMCzaaVCboPiT4AErNmFbh1oz6cmyb8vePbUpz61NHCbqE0QyhygSXyXN4mmePSjH536gGduws76TPrp+K53p512SjSd5C/896CSA4Dd5wF7Hn9wHP0zyTdJmeDe6bdRJeisQf3LxyF+9lsggLvfJrW8Q/YFQHkz42TcDeuMpBoA4RzFnVQkeeTAXVQ1CEgKlOLCi83DKWGF19PEMlmNFIoH24EVszeLfWjqdxI9WYAKO1lhSH5Bf0jDL1OjxeokV+mw6XwvCh8bSKAccNm8SJOioDGvW3ASh9wUcLMjAFQbxybENueVXDWMhuBhuyd6HQt/47wVxWqquTc5EIKTzjOI1mvq3LNw3ADuhkYxz94TMS/r4VXpiodOyjYwkVqvmSDyZTmwypef1BVqQtjhBVUhKQh48dQFzKhLylbUq9JHnyU7JFsEmGUkeFWPP+zzAAyFxEYmJQCKLuCJ8rxtezYfeAPu+eSAji8Byjifd9556XpRFr3gbAJBFTlvHXLAYdfN68Zn47l5/ySRg65T8JVMkzSShHC33XZL94R7wV6Ye+65Z+rrsPOM+z7uXWKQ+9fKMVo1CwRwV7NX6U97eIAVUOV11t2AMj4Tj0opgQbhTeN10Sc0wxQdwBJo5975IK8ROOljLiGrs68kXxQs+nPrrbfWeeihx+K9mYgEJmVJFhOgrIRMajhsHOtczY46AplFm1k5SOqxsiE15F1TR8xHKwBHUk1JU00CtwmDx2yCQtXw8E1aeTPGqBOTNpqieG3F301GTTgceT+k37OdezX2n8ytUv5nAHd5W1X6pLoPlsYeADuf1908gIAQ6PKyARWPW/IE6gRvSdHBmwEYvC+8K5WEn/kLL8v7JhXr9RLr6DOOmbfJDpPa3V5ADuDaPxJVVWwmN7K5ubzR/PPUN0rSAjifx1PLgFR/pTgZ5J+f66cxETh27iaBO+8DrTqnQTZn8XzkgVYHbAM0e+8H9wX6QoZpE9RZ3j8OBbULSkcMIFo1CwRwV7NX6U+rRf2KV7wigWYxGFj6ACU+CLx53jhMD5kHlLdpGSrw5P8tk33G8h3P3fvyPjVK8eEucerSH5EkZDkuq3AY11r6YCU+CJgFBKkV1Nvu5W8dQpD0CU94QpJOzrVMRzEBOWnt1EG+h4Jha7at0iS38NJNkk3Zutgf8j6Td28zOVM50XnPdU+4h5qYyPP+CIqiZRQgo+2PVs0CAdzV7FX607xL/OKaa66ZdNOlvzhjH9xvv/2Sp9qEnn2QqWQ74p2loAOHQY3XibKg47YaQCkNAlMAbSIQaARk4zTUmcClYGeRuhjnmMO+67oFHefSbg/77qTel8lptYhbn3TQelLX2OR5Argbsi5PV6Epy1Fc9EJrvDX0y3bbbZe4Zt5qVS+1qs0EHwUkLfcFCoclPumbVYbPoj9U+mtCa1/sP+8WXUTpYTKoy5sVjESZkdYJfFoZKDplldeEmql4TeP8jsJS46WpWj7j9K0L3w3gbnCUfvzjH6ebUyrxJOo/NHgplQ/Ng5VOTkmB6x/k0VY+6Dxf2HfffRMQozbK8LOADu9sVcSrbrrht3HKFEd1TRSuk5xOoo/gIkdBbZhpaObL2s+Kgy3UTDfhFGWrZY+x0D8XwN3gHcDjI7eT9EHNsJAa7xrXDBjtddlk8+Ar3apOi4mCwqaMd89b5flRitA747SbbJKylCRQGKwuxYaYBnpILRY6cglW4hhtblLdqV6M1yTiHm22xah9C+Ae1XIlvodLFSTC5e21114lvjE7HyGPE3iiCZYI02QDgmR/Ao5ogiqNDBLPatlurJrkn6mLyPAoVYaloFfpu8+aoKzsZEjK9Gw7aOszesSELlAbiTdVR/zuzwdwj2a3Ut/yUAm8uEnVvaDuqIvbLNWBKX1IpiDqgRdowmqSa8VV82RRBPaPrFpvw+RKh602uUmG7ryphvMXFKV4WajaZfc/Xb/nAR/P/tGqWyCAu7rNKn9DqrVKdXTBwwJmlQ/a4i/wgNEkrlnqdJMNRZJXxJNsMkpDtainIjEHP94k56omiFoiTSVmjXL9k/oOakp9dglMJsmF4sg0Yd8A7ias2nNMJTZl6uFfu7CU7el+5T95sLJGSQHp2ZtqdpmRTq9sq0lx1JrOvEDJSioFCvJJsJkvE3LUa5IxaP9JWY3Os5AaTb2gJOqQVDba6BYI4B7ddqW/aTkvk02NDDxkGcVD6YO37IOASbU36okvfelLlXZCr3opZHx4bfw2ud24DeUCVCQMNRWoxKHLqlW9UQLKoAShca+jrd9HGwpIqpVTrEbY1v62uV8B3BMaHQ8rUKDlneVMMan3z3nOcxKPme/CUreJxQ548opmSd0WmKujSQ9XxxoHzSNsKkUfZaCmjBWYsr8LIe6BFrG5CI291VEZ1U8dYzqrxwjgntDIWsar2yEBQ0KOAN4sNeADqO3yojKd7L2mFBoAlhKE/pq0rs7GE1TbRWIOmV1TTeYgmajiW6NSPE31rYnj0q5TGVllNFF0rYk+t/mYAdwTHB3qijztea4dtyfYpdpOxYvE3VpV4DGbann98BVXXDFJ61S6q7M5PmUJz1DWp7K0TTXyw3XXXTdt+swjndWmIJeJVvyAhnvWnJZpjFsA94StTtMskxDA8UpnYZkM7Mjb8JfKx6rzXOTx67xGgU9BRPI9FEkTGZl4Z3prPLRsTJNS3c0x3/Wud2UPfehD03kEsGeRPiD3o9jZeOONk1yzqaBv3ePT9uMFcE9hhPCnpHL47q4vG4GNIKTKdzhb4G1SksEn7bpOYFXo6eCDD04ZgkqC1hGQHDT8JhpxCEt7mwbbDX5cvTGJIUWRnYsuueSSpOeWVausrlR1dVNmLfYBpE1IVhWKfjW5UfKgcZzl9wK4pzC6pGfqHed1j6vWdp5Cl4eeEqiROpLk8bqVL6WrFuCjVa5TnUETLmVcre26UsaHXliWpZKuZI0qCMpOrNKAv0kN4KtLYqWgzKxjSRbKNy1Q6xuf7j1qi9762VXO2bbPihcIHpsA7XYTrT4LBHDXZ8tKR+Jpy/STai1BpyowVDpZjR8u0h74SkAkccVuJnhaFAAFDapB7QxeuC3AxmnOSWaIYrJSoX9uKvBZ7CedOM8euJ544omVMkBVADz77LMT72+jC+BsrNVFyUHbT9eT70hjtWKnHPLRrjfJUJJsHvawh2V2EgqKpN4RDeCu156lj4ZCuP7665O3aikpeGMZ3YUmuKTWsy3TAI/NgIuJRYCWt+V9/08XbUOFUeWBNi+wbyQVie3CpI5PqllR6L+EGcv+YQ3YilmQutntRgam71ARFYG6+Lu64fT9GnmjQle4e0qZSW3zNux6xnmfzQR27W5jIwqriGj1WiCAu157ljpa0Wu1aWsebBPUa1JGV6pz83wIz3z++eenLbyAkh3FpTHnrXhtHlhJMnhiDzGQ8tkqQTjH470BNDy6FPdJNpME+kfQVd0VNJdm4rXCkFTy1a9+NaMSArwmqiI4D/p9mWWWSZ+xIa+VRN4UobI1Gs8clVa0a/6ZNv+0CqLCwd3b71NW60Io8TCNMQngnobVe84paOMh9kDzYvOtzoog2POVif+pLwJsAAo44bTRI0XgGdQp3rngJW4ajwvcrDTKBi3Z5jWveU36rmSVcYOEg/o433tUJgqFmYBkhWp2zVHljuZ78eLFS8Ha5s2DwLr4ntgG/bYs2t6WFyUzUbAVLr/KRNd7vEn9bTxtvmyHHxnCJvRJrowmdZ1tOU8AdwtGAijaaMEegfhiATHL7flAcZJdV4ZUDWWcpeCj+uJlaykDfFSAMp68dNQD/rcMCFtq80B5b9PyQAEnGafrlq3JqxSQZQtB2CIol/ndBC3DtCiZzMdSjMC9YL9S1RXRLXZSKmOr/BiT/onKokknn9RnE1pTip9JX1tbzxfA3aKRsQwno3vWs56VwFv5S4GdfHk+ja4CXHpj9UAE2PSJemSUHVyuu+66FOyjNFCTWu1sAcBiK64ygCVbWHbz5qYJXiYfgUq8tB2NUEC0yWWAuvgZu76YAOfbEUmMwE424h/klWxlZ/Q2NV7217/+9aR1p/SxqgLgbXI42mSvOvsSwF2nNWs6Fh532223TeU/LT3VzyAns2yehJrCfpmABXCiKXhSVgLoHOA7ThOgNDmhHQAyXbYa2r1JLuiJ7bffPunCAWYbGg5a0BGNwTb5CqkIzPP9DuDENUwE8zVeK8CmNsmTgXxXILgs1TTfOUb5f/cgFRSqx0RuFUU9JDAbbTIWCOCejJ0rn8XyU20TXh3elKKCSkOSRu7t8k6LHmrlk9zzhfw4fkrLtxfjDjvskHhsvDSFgABcXanKHnySQfreFVZYIcnG6L1zj1oA1EpD3WrFmBR7quM6R7VP8XsAG0WiJgttssJJ84F1/v/GEPdbBXTZxKRtIjcWgrR49nxPyUnZJT+Pe8DY4d/JPfHZVDC4+WiTs0AA9+RsXflMAJrnC8R4eh58YIb3lNxC3SFINk4DDGSIAn8oAIFHadg8Yjpz3j/grHuLKR6n86rZzWNzXhMVvS8PXLlWq40mCz2NYje01RFHHJHK1h5yyCGJ7yblBGA5QA/7KVOWvrtqYyseLo+WNtr4oGwk7Fx88cVLJ7yqx63yefEMY2ECwWNbgYlZ2BDaCqB3xVTl2PHZ6hYI4K5us4l/g6SKp03FIbGFLA54S0gRJAOwsvJsfGv5Skfr8xJkeLJUHGSG+EgFlGwZRR2CgqGLzqvUOSZVi2JRalPjt5t+IPG2qiXicm1nZVVhUuLN2b18VO13U4PEWxYkNalsueWWyds0BlYOADuX+vWCN05/3OxBEzn9vN2FJD7ZSYbNADj6ybij06p49MPsZKI2gXIOlGhAiQjOonqsgujy6y7wNawv8X6/BQK4+23S6nd4PhJBFO4B4DyvRzziEYkvFsRSE0Lwy1IWPw0IZWZSZQB9FIhAo0CSam2+y9ulJbcEB+447kk26gqUA/CWpYhDxuuOy6fXfQ05XeC4b33rW1P5WoDN+zTpUQMNAm+fAXTFJKVx+0bbbmwFeqlugLixNQkKHuPj0RfKKVgxmQDRYBKFTACoOOVkvUdvz6O3ujPJ49F59ygQ95OgtPuEowDE5wusjntt8f35LRDAPb+NWv0JS3deNO8U7YCP5oUBY8E/fKwH+3GPe1x6yKk0FDQiN/zgBz+Y+Mq21IO2YuCZAj+AhFdvYyMPBI7AWl9NnoASmNN0F7Xcys8aj6aKiQFgdJbJme1yeSI6hYeMi6dOAbgSidR7OeeccxItpXQAFQiwl/wFpE0A+erBqsf94v+t2MoEVNs4XrPYpwDujo8qL5DHynOiQpAqbYmLP0Z18FolsaBMeGCW+XhW3h/AbtPDaEVggsFv4/KBN2BpU/YdO6OlrAjEHKxagDMvVcBQLfIiTcJbNbHWHSMo3rZWSMbe5CCQeeqpp2YHHnhg0oKr/y7ATZUie1U8ASDbWo4XrX8meQlB+GvUmaxN9WVQbbxrq7xJqJmK1xS/z22BAO657RP/OwELmDwkmVgh4Ntx8BJveJA8QKuDNqgWABnvFWjLmKRv10/erTiDyRLQiRMAbyoQnuykCyyhQkze6BQ7z5x++ulpAiQtpD8XVMWTWyGQNPLEBaftymPlYGLPFT4TGP44xQgWCOAewWjxlXotwFPM61sIsuUNdWK5j+rB506rjgsgJIGTzagvYgl5wSn8sJgC7xWPbfWjZAFds8/zgKOFBeq2QAB33RaN41WygEQbXKsNBXjWuT45P4j0cvsU+n98K435JBtNuXMKnAqaksDxSPP6IXTN6CiUA7BWl0UTFIy070mO1MI6VwD3whrv1l2tIB+vGq/Nq+1tAJJk0DJfEBCdosQrL7jphm6QtcmbRtvg2yk0euV2qJ4lS5YklYnqfqiRogKl6X7G8ReeBQK4F96Yt+aKea42DhCIJAfk3Q5r6l/QsVNy8G4BalNyQaArsYSnLxGJp0/XPteGvrht+mbJKT7bZDBymI3i/YVjgQDuhTPWrbpSwS+BMhSEYB7KpEw777zzUqo82kLgD4c8qMpemWMN+ox+SGSRXGNCkalYVpYo85MShkY+klMGWTfeq8sCAdx1WTKOU9oCsjEFJHmzAn045CrUAh5c4E+lPduCka7VkeFJHnnSSSclmd9DHvKQlNpepdIdqaCSprInSerGLUdQ2qDxwQVngQDuBTfk079gQTt1UXDH0u6rFq8C8kCRnpp3S09N4jZO+VsessCjPuHcSRJH2XJL5qHMVJOSSSBaWKAJCwRwN2HVOOZQCwjc2YRBEgigVHd61CaJSPan3Wnw3rTUgwKccx1fcg+ts6AisJU9KGA6amKS67OCMJk41jjXN1e/4/8WtgUCuBf2+E/86vONZPHHKttVoUgGdRZFgoNWR0M2oBTvYunbQd/J30ONmEQEPH1XfRdJNuM2fULlyALFw9fJwY/bt/j+bFgggHs2xrETVyFtWk0NICk1fBQqYtCFAkpctKJayquSDPKa51Kp4KNVIZT1yDtWVKnsVmyD+lB8z2RkAhB0lWJ+ySWXTDx7stif+H32LBDAPXtj2torwvnmdbZJAcf1tnsvFMctfdu+kDYcUEGxt7gTXbi6ItQisiD9lDTTRKEtWaC8bpz3tPbL7LVR/D0bFgjgno1xbPVVAGjFryhIbBRMs91UA8C8Z9SJwkp4dFSKPihtapNe9VAUXEJnqG/dVKN+URpVzRJqk9j1vClLL7zjBnAvvDGf+BWjRBRjQmOgStSEbropsASgcekoC3+rJUIxotQtQOX1N92oVQQ9pcMrpxotLFCHBQK467BiHGNOC8hCxPUCUAHJSTXp6TaHoMmWsKNin30u7RI0qc0iTFJqseDSbV1mEqubIpqUPeM87bFAAHd7xmIme0IOh65Qs5o2uukSp0VQJOlDUdhVJ6+RDbiLksHi55saAIFQ0kCet3Kq0cIC41oggHtcC8b3h1pAYo2CULb0UpNkktXy0CBAm5cvIEqWZ59GgUvJP7jtpieRomGk6tsJB+9+zTXX1JLpWTx+/L6wLBDAvbDGe2JXq4IeioSHC6yUZ51EcX6qEUFBKeeScrzUROF987TVzn7Qgx6UAF2izKQoE/awBRh+fd99981++MMf9o2FPipkpV5K1WzSvoPFGzNtgQDumR7e6V2cjETbZinFquTppBqpH1ri/ve/f9qKy6YGxYb3JtMDoCSDUu4n5XkLVEryUWMFbZTX9M77Jzlpr732Srx81DnJrRI/B1kggHuQVeK9sSzAY7TvIRWJfSRJAZtuwFe97GLtEjvmDCqvKkCoMJXtxwRN0Tj252y68bpRNLl+3CSjWSFISNJ3NlMaVvZntLDAMAsEcA+zTLw/sgW+/vWvpw1qVe5rWgInuCjJxv6J9qfEZ/Pw55P6ybbEO2+zzTapsBTwth3ZXNmWIxuk8EUqEyBt30pBW3VS8O8mkDyA6qe+RQsLDLNAAPcwy8T7I1kAFSEgueyyy6aND+pKax/UGTyw1HIZkuR+PFaefhWlyA033JA2TKB62WKLLTI1tZsEb163iUYiEjpHIlAO2MrB5r/LAG2yH4PsGe91xwIB3N0Zq070FF0hzRsN0TRFgu5QIIrcDyVDrTFKQ+1IzuEF01tLzmkqy1EA8lOf+lS2/vrrJ5BebrnlloJ1Dtp+2hzZpBItLDDIAgHcg6wS741kAd6vEquKPF1wwQWZolJNNMWg0CGAFgBSa3z3u9/tC/ZVOTc1h4lgu+22S+D90pe+NEP51Nlw2nvuuWfytnn4RaAu/s7zFmC98MIL6zx9HGuGLBDAPUODOc1LoSJRnY/0b/fdd28EtPHSJocDDzwwW3vttZO3bR/KuugYckXBwx122CFxzkAc1/yHP/xhZNNSjphoPvrRjyaVy1yAnYM34KZ9b7Kmy8gXFF9shQUCuFsxDN3uhOW/WiDqgqgPYh/Iupua1jxgQUTgRx9OJ95EUyJWkg7axO7uZ511VipQNcq5rDqoW0w0OTCX/WmCihYWGGSBAO5BVon3KlmAlE7p0rXWWis77bTTKn23zIeBtnofNOH013vvvXcmCNqrgy5zrDKfEUAkLzzxxBMTFWOiAKI/+MEPynz9Pp/RRyVdrUQA9v3ud7/SAI63b4puuk8n44/OWSCAu3ND1q4Oq8NxzDHHpNKl1CTzyfCq9t6koKKgzQ5U2EONTCo5RdCSXM+uOvai3HbbbbPPf/7zVS8h7YBjRYLfNvGU9bjx3AKUTU1QlS8kvtAaCwRwt2YoutkRNAJAXXfddUdWdQy6cp6mnWNQFmgGQU888SRKwhb7w/sGuoKVeGf9oE2fb+MFdVmoaorqFDJAGm70Sw7eRQlg/l7+k2JGhmWkvxdHJH5ngQDuGbkPaJc94DIF1d8QLLzzzjtTYI0e2O/eQzvgpAX6quide83kuzYmeMlLXpK8UTJAx6+jAee8zgl99gtf+MLsxhtvrOPQIx9DAJRMUNaj+icf+chHMpsVD7Ih+77//e9P9IqJja3zZnxOOeWUpDmfjzZBDVnFON44TR957e4PAViTopffvRce/TjWnc53A7inY/faz0rOpoSqpT0++J3vfGeSyR1wwAHZwQcfnOpfHHvssdkZZ5yRqd8BCIH5qM0Dr0QpEHvxi19cW7EmgcF3vetdaV9KnrZ62rzbNoCLiYkGW4amQlXqitx00033MSFgVnNcBifPWfbo97///fv0H2h+4xvfSAWvfEay0iDP2845MjvHraXCfvpw7bXXpvNSzlx66aUpaPqd73wnFbwa9xz3MUL80bgFArgbN3H9J/CQqbYHhGUNkq3ZFmy99dbL1llnnRRQAxibbLJJ9oxnPCMpMDbeeONU0hSg5J/xf4r7K7RkSd67P+OwnueyPNTBlltumVLFh322yvu2GKMakU0oixE1UpfUr0o/5vos8AZ8JivXr1SsyTL3vBWSYldgDJStGMgkeefFZuK77rrrkkdNvZLTI70/jel8tEx+XKsp4Hz22WenMUUzUflQ4MgqNeGgtOjsJUn5nQ7evSJhyjXh4U2c0v+BPaooWvssEMDdvjEZ2CMPpQfd7i1HHHFE8tbUmgbYakzjQ218C/j22Wef7KCDDkoe8dvf/vbkteJWLfU9mDvuuGP2nOc8Jz2wvuvBFghTuQ7lcdFFFyUlRHGJX+yUokjOAXAEJgcVcip+fr7f6aQBhT4AFdfhb+DW1iaZhupDEpD9LYElu73xjW9cCsK8aHQIOwH3QWVtrTDQIcBz8eLFS7+bA7gaJrziQQBqssClK1crVV8pW/tosqO4Qw7W7hHcvGqN/m/77bfPXvSiFyVQdw8BbfcBELe1m88KxO6xxx5pyzn3nHuvyNe3dVwWSr8CuFs80h5MoPiTn/wkARmgQE14wJUkxS8DTtQHNccgYBh2eTxH2YYCbagUnuOqq66aSo56gPfff/9UoU7yiKV93gAIfpcnaZLg4Y3aXB9VioxFqwBbjJH6mRi60IAuSgrvzZvlxbJL7m3n4OsnW81V8c8mxiaA3hR4KhSgXAzK4rxNdqgx9Bc6RUVB33VfmPiMqXFSKvbmm29OdnYvFScA9BOHwKrG5CDb9aSTTkr1wm364H5wr5Ey7rzzzmmjZQFXMZNhk3oXxm0W+hjA3eJRzGto8ILwnThfy24eEEmcBwhtArCrcsBA03eAsmMABlSJwBlvzEOLxxUYPPfcc5c+8F/60peSZluyDU/PcUZtdNE2FVDK1LZevNaytMCo56z7e8YAxcOrVTRqEGjnAG7VM2yc2BEQs0f+eT+NwyGHHJImb303Xueff36atIG62t68a8fGrQsYA/U8CJ3fG8PGyfvAHBD7LHAX3HZdQNrEjstHq7gH0VhWFSihaNOzQAD39Gw/9MwCjfhdoGkpzvvBZduxhZc8TlBx6Env+Q/n9lCeeeaZ2atf/eqlvCgaBTCQxeWbI5hYRmlAgh6aZ4gntnz/whe+MFZq+Sj9GOc7ORACO+PC2y4C7qDfxRyM67AG1AWN3/GOdyS7OIYCWkBT3RKZo6iQPEbBi+chCzTaUWdcympQv0wCJnSrOv3CmbtW1A4AF2uJNnkLBHBP3uZDzwgMrr/++rTVFt4ZLaJuhiXvJPdrzDtokvCwChTytPCh+VJ80NZb+ffm+mlZDrwENV0f8KHX7mpTkVBwuEwNElSGyWrQWOYTATtYdYg1KP2aTwB4adSYcUBrUdtM2uu1MrPiUrkQcItHmMg//vGPZz/72c+6OoSd7HcAd0uGDW8pS1CWHq6XmkCRoVG92jovC6Uh5RuXC7j9jvuuwnMLzyblAAAcuklEQVT6LG9esNQDnwc2u/zA5zvX5OBa5ieKybji9udrtlgrHpOnS4LZBqWN+itWZDh9pQ4ok4xlmwPK89m7S/8fwN2C0XKzU1HgefG9++23X1oy83Da0HiDlsy4XLQG7pvyoEq9aN47pYIHHfVDUlcMuLXhOqv0wUQrgAtYB2mwi4Bb/J3KBNDNFajUD8eXxGMSx51L++dhe78YYKzS5zo/qw8mH8FPqyfj6p4Q9wjwrtPSg48VwD3YLhN7VyDIAyoACbipBOhn29jwuRJHLJXVDkGd4KbnAhIrBqqIZz3rWYkaoS1GjTTBx07KZiYyVBF5XRGUy/4O6PPJOe9zkSohuyMRJOcT9OShK1Q1l53z40z6p4nkyiuvTNdDfYLHlxNQVxbtpK+nK+cL4J7iSAkyUnF4ON30daaNN3lZQIu3SWNMgSJwVgSe/Nw+d9xxx6WEFIoYgI/D73pzraoTmmR5mQCWLcp63rxulJOCWb3NcdFJJnE8sizYohyz9/Nt+dtY05Hj4Hng5I2UKdGasUAAdzN2nfeoQBvgCdApOmSvxC41HrMkH1XzeNOokHyJTB2B75Skg9PFaX/wgx+sLS2+TXZCF0iDJ5nbYIMNEmWANqB/ng/I0U42hsg9aUFLvDaJn6QYsr8uNdSeicZqzGQmaNnllVWbbR/APaXRAdqkcLxtS8suNg+qLE6SRWneeb1q8jR/r7DCCtnLXvayJGNrC19ft5153+gC3iXgtYmEHedNZgB4LvqEEkUWq8AtgDvqqKNS2VcBapRUF5vroP2WTGRVYfJBB0ar1wIB3PXas9TR6KEF6njbvJImddmlOjTGhyztAQ7PGxXyhje8IdEHAnA8cjrf3BMf4zSd+SoQ/vGPf5yCdBKKgDhwNkkPAnH3gAqCqCdxDoCPMx6WqNMFQ0j+Of3001MZBnSP+z1avRYI4K7XnvMeTZo0GRX6AODNgjci3V5Gp2uSXSdBpC2ytXkHpOEPmJSpcaTG77LLLokCUehrpZVWSkBOMYIWoR6RASkrNm+D4gb5/7X9J8UQ2kR2p/udfDBafRYI4K7PlvMeyTLSjuRqgajKR6UxK009FRUIBenUyYjWbwFATPsN0CTUCNjmPDi6qWtxjv4rvO87JKRLlixJ8sfXvva1KQ2/y5PRfa9uun8FcE/I/gJQAniWxkBbUZ9ZaugQaezAmwcZcrDBo4sCscpCJ0gjp8pBoZCEdpkyG3y1WdLq4+wF4JWLzQOxwz4f75ezQAB3OTuN/Sl0Ag5YGjOp1Cw2sjUbN+A1AdEs0EBNjpPUf6svhaW6UhFxFHugitT6VkFRjkJ43aNY8b7fCeC+rz0a+4tkTJRdajDN66w2daqVADVBSeGPNtgCVl84b/p9Gaiz7olKIhIDoVEfVKtlsJXi3WEWCOAeZpka3/eQSlmmaa6SJl5jFyZ6KCoJOmbJRVF8f7DpVXsUjFSyddZBmwVUGKScQRWiiKKNZ4EA7vHsV+rbkipU+5OQotDSrDf8vZK0JI9qkkS71wI4bnp3yhL0AengQmmkoVQmipSp9R1tdAsEcI9uu3m/icuTmKE0q6w6+txZUpIMM0BenwQ1RPIY7V4LkMkplStYpzTrQmqcFmV8lQk47bTTFtKl136tAdy1m/TeA0rGkHyQ7wfZ5aSKe6+q3G9S3snddtppp1Qgqdy3Zv9TNkqQDo7ftjHCQmrksOp5C8i6/i7UYGnr+ARwNzgy0rzzTEIZdAupmbQU2VdwyC4t0bK02jrvvPOSdntWa7fMN87uCxtBcGaURvB3tOoWCOCubrPS30AZ8C7skEIStZCa1YXlsJol6pVEy1JgWoDOSkRNk4XalKxVy0RJBHVeolW3QAB3dZuV+gbgsgO6TQfot6e9CS51hwQZ+wRKiLASUGK1KUWD4zo+74o0sMubJpQa8BIfsukyJYl9IxdycO6iiy5KgVnJWnFflLhxBnwkgHuAUep4C1B++MMfzlZeeeVUZ7uOY45yDAFSGXkkiRJ/KBkEDcmyqFzsWNKU1+O4QEqNZoHZhc5pHn300akYlwJM057I3UuyW2+++eak/LEDkzGSpt50E6SUdKReC61/JORUt3gAd3WblfqGDDHerWwxnta0GlXLTTfddJ8Ue6Bx4oknJq6VbA/X2FRTbEodjrbsn1nlOgGKlYPV07jgYvJUiIuW30ps2hUTTaL6oa6M/T+l3dvT8hOf+MREwNumIVajNsJuwyRW5b5ow2cDuBsahauvvjrpmJsGxvm6L/HBtle9QGFi8eB4YBUAaqoJTOYa9l/84hdNnaaR45r00D2UEOPWXlEdz8YJNoGeZikAE5Dzm8xdG+9XOYbPfvazmW3llOdVyfC2225rxKb5QTkz4j+cm7Zu1Zf3tY0/A7gbGhXBSFrdPfbYI7vmmmsaOkuWKRNr6S0A+KpXvSrV9y4mddihZdDyF4jaqcVy9Zhjjmmsf8qUbrPNNgm0mgaDui8CjSSYqGgW2wK0Cy64YKSkGbQZhY2iUnXEFcqsAIZ9RgKQybz3vpDxaiLngasT32RTGkHQ3r6dEnOiVbNAAHc1e5X+tB1uLItRBU0UELLUlaGINxUAVAfCUneLLbZIVdg8nMMAQhLQl7/85ZSWvuuuu96HRil9gSU/eNlllyWe2yTWNc/KzkQ2NwBmXia65z3veSmpCIDjZ02cZWIE9mOUeGIiqEvPT25qnFFdXpwF9na/5TsOuQdM0naIl8Vq3HHZg4KC3s+vFbXVZFP6wYbJ9ta0U3y0ahYI4K5mr9KfVlTKprAnnHBC9stf/rL098p+EGAILjrH3nvvnb7Gu5bcseqqq6adV3rpER/ywEoK2m233dJDimssAzxl+9X7OZMLKsYkJkDapWbbrec85zlLa2arnZ3Xz/bTjubqq19xxRUpAMzDBZSDPF0rL7vbHH/88UMn1Kq2UayMY2ALtBxwbWRx6KGHLlWtmOB5z5Q9PuOzVhKDGrrEZ1Am+O8mm9WXzFF0Xdf21mzSLmWPHcBd1lIVP3fmmWemh5yWuYlCS4Cb92b5bRmeN16dh4+nWARkYCIIpIB//pD7ueeeeya+M/9+3T+pFkgPFZ3Cq3ap2TuRDj0H66Ld/G5PzYc//OFpW7KnPe1pmcJRpG6D6mq/4AUvSNplxxy2EqpqG8krKu0B3Mc+9rFpEjeZ46zzZtxN4Mrtoml41WqBF5v+iMkInnIE8M+yHJts7kX37bLLLnuf+7fJc87SsQO4GxpNlfE8BLydXi6xjlN6+DyE0qall3vQeH4oCaBij8Piw+cB9h01oAGp7aSAPE/szW9+c+Jt61rCF6+PZ8UDBH4kZ7fffnsKfNmr0krE3/nL31YNEpeAX+/LRASsel91AWGx334HiAB5GHD3AjmZJSrFZMizFtTMV1smAC9UxSCPvPfcVf5mD96rYJ/z5+fMj4Ea+9jHPjYQINne/UBTvcoqq6R7x707aPLJj1fHT1QO+3pGFGEbtDqs4zyzeowA7gZGFgDaSIA3wXsZV5HQ28XeBx/PKbnG1ld4boCiznMRuHuPYasx0i8aaw8srrzXE+v9zih/8wil++uTACoQV8oU5wts1GfOX/pglxQPMo6193XGGWdkVjJAKH/5W1DNpMAGxdcXvvCFRAsJhOUvHLDfTXgmOpMf6iB/+VuwDH+tP4qD9QJ0mb/ZFIgqKIWaMpGKPzh+7/iNYtfe76ChFHDSN5NDXswMqNsxXsyFiqTY9EMg+/nPf37S9YuT2ORZ0JDtmgRTx9ZPe24a/6YniuJ1z8LvAdwNjCIPkE4acFsaN3lTeiD32muvtMyVjSczz8M7H3B7aHlmwMXnV1999UYoHZ41gM7BjoeVv3iyVV/5ccb9CTDwqzbpBbL5S3xgzTXXTEArScnfo5xrkJeOpsIdNwHc7jF2ds+pyJgri6xS3IsCk3Pdh1Y5JjScOZ4cBQdYm2wmywc84AFpc5Em6MQm+z7tYwdwNzQCvEMPrwSDQRH8cU/rgeS94bJ5WrmKhFcHaOYD7vz8OecNxNAUdTeTifrLAIVKw9/AS98BBQ84f/kbwPCaedC9L541e6J7vHjbfrK1FQ5PXTA4fwm8kvABMi+ev5ff1cuQfKJvambkL7QRtYMyBTT46oqMAtyDvvOSl7wkKXiaAG7jZhUhmGqyYSvNvUcrXQaErbisQvJrdm802b797W9nyy23XNpQ+Fe/+lWTp5q5YwdwNzSkZ599dnrg7ejdxOYJuGMBLwAhfThvPKUcuIvByfz/e39utdVW6fOUH3VTOs6V7/5jYiir47ZiQTf1vvCi+iiBpPiSKGPS6X0BA0FctBDPP3/5myKDPFHwlKY5fwmgokn85MFKmBkEwvO9J5FFDAHfbaLg0dMs0/Q3BdzuM5OXvpmUXL9rkQRVVtHD87bJg2P42WQzeecedxNOQ5N9n/axA7gbGgGBF6oDnK3gYd0NX2m5nz+kjo/PzKkSXDdwE3zijfO8ikE8nwWkgNsEAKi8V3fjVe2+++6JS++aqoTXmgd75wNqGm+eKrAD2Dx3JVxNHNqGG26Yxgbf3BRwOy6gpjCxWjjuuOPSasR7ZeMX7gGrNdf78pe/vO7bYenxnOeSSy5JwB0c91KzlP4lgLu0qap9MA9IkeyV9XaqnAGHqcqfB4xGGH9JBrbpppum9zx8Ei9wloKCZIkAPC/0xPsUfLOMblJHS1lhByATStcq4slIHeZx8xRNnPh6qwnetElaYDMPDBbHk+abDYBVU8DtfAD6iCOOSDw9x8HGzb21QJw/fxX76Hdeu4C1gKWknaaawLkAKArt3e9+99L7sqnzzdpxA7gbGlFcIfkX/pmXVXdDG+CDVR9UxInskHeHw0WXkPsBctwm4MS3r7HGGkl3rEIgfhhgowryLLu6++h4+iWz0044JosuNVx5zvf2etyKh9E949cBHOrFcj+fGHuvE7CrQS0xq7jy6f3cuH/zZFEQZJ4Cr7j/oretf5Q9Jm33ZXGS4Qzw0k30TRdGQ8mIT7ArG0arZoEA7mr2Kv1py1MlTXlZTe1qzcumuQUwPD0AjI7g1Zk46MdRJM5v6S4QJ/CGBwU2kyj6BAjQDTz/rgWg2MnmtsDFpIcCQfsAPWohQdayGn3ZiGqeGKsmgZsnLbZhFUYxpLhVEZz9LmAri9MqQL/EN6zIeMAkpVLQm+acTeL6IZNTPZto1SwQwF3NXqU/LQAmuMVjo4RYqI06g6yOoqSJ4GeTdqVuoQTZaKONksoEZ21cR2kmLsAvg7FJ4M77hn4QoESTFKkZ51bLREDbvYkPJ380IZGWotPyVvxe/l5dP+29iaazWiyjeKnrvLNynADuhkZSYJAHg2dEXyy05qHHl0rmAA4UHE1kZjZpV7QDLtZYjgu21EUoLPtwNm0HXrXVwnvf+94mzTPWsWm42cJkhtqJVs0CAdzV7FXp0zhDxZWUBG2iQmClzkz4w8CJJ4Ue8AKCC7nRWFPvyMRsMqbAxmIfVnnAsa3tM5/5TKLQeP7iLNGqWSCAu5q9Kn2apyYo5UXXvZAajxs9gONXfGmhN5QFPtlELg5Rl9fNu8Zj02pLKlqyZEkKPopzuP/a2gRNBU9x6k3kObT1uuvqVwB3XZYccBwRfNl71CV56dUBH5vJt4AG0OZlooyiZakErGqC6qxQVdTRKEaOPfbYpan5JgYJP21W8EiYMpmTUVK2NFkTpQ4bt/EYAdwNjgqvioabDEwEn8KjyYBPg5dS6dBokeuuuy7tpCJz0HVHy1IaumqDasuQD9bRKEhIDCVcCfRRuzRNxYzbbyUOpNOTVNY1gY3bp659P4B7AiNGgofbtMt6m5evdZmCt2fJ7prJzKLdbQHyQSsvVfjq2q5L0NTEaMMKElTebNsbtZW4x+te97o5K1i2/Tqm2b8A7glYX/q7IvYi6E2kv0/gEiqdwhZatqQiBQQm0e62ADkkmkTWpU0EhiXrzLK90CI2TbbykFUbNMloox3APZrdKn1LMoN6DDbmpZWd5QdWec48scK+jMXkj0pGm9EP22uRxy3rssmU8jaaD0hbaaBIUCUSyBYCddjEWARwN2HVAcckzVJHRDpxm2VaA7pe6a18N3Mp7jZRiHZfC8geRSPZG1RW6UJqHBhZmmI+MkijjW6BAO7RbVfpm4Iwqs1JnSbbansAqdLF3fPhvPCVFHcStVgG91tR4PZ73/teKl6lSNhCmtyUtFVCQMakMrrRRrdAAPfotqv8Td6WQvfqRMxaYR2yNJsR2LtQUkVdOuXKRu7AF9ADqBJySbTSQmhKCB9++OFpazQqmGjjWSCAezz7Vfq2B1a9C9JAwRme1ywAnNWDLEmb5e64445px/BKhlmAH7788stTWV6USdkNJrpsJpmcKhbamLguKWSX7TFu3wO4x7XgCN8XqKS7pTRpc6JE2Uuz5Zgt1HD4Tdb2LtufrnxO9qA6LjZPnoX7YJjdySDFPGyYrGZNBCSHWar8+wHc5W1V2ydVmLNspDKhZe0y30fSJaXfVl12jZ9rQ9raDDgjBzLuKvg99KEPTdSZglaz1lBoyhtbWSi2VmY7vVmzQRPXE8DdhFVLHJMszO4kst26Gqyhjtlmm22SvI1SonenlRJmWNAf4XlK+ZaopDTArNWlplu3qrAbkw1FFgIlNKkbOoB7UpYecB5cn6Qc9ar32WeflJwzbvnQAaep/S3abJl69jXEa++2226dq7Vdu1FGPKD4AAmlCXzbbbfNTOizUElRroLVGM26bdDkL0SrzwIB3PXZsvKRBCY9qAJ6Cu4o2l8sZF/5gBP6grKhAqyoHp52viHuhE4/c6dxH9ib1ARum7dZ4LvFOiiM1lprrSgy1sAdG8DdgFGrHJKHffXVVye6RJlLUkFccRs10HkGqJ3MedoK9cfyt8poD/8sPbcgpd1oTOCKdHW1Set/+tOfnlYRdj4qu71bV693Gv0O4J6G1QeckzRQoJLXRd9rx/Drr7++FRF4ASb1R5TiBNo4WcqYpvclHGCmmX5LJUmUmdKvNt+g1ulSU0BNdUKrMWntystGMLKZEQzgbsauIx3VTS7yTiqorrINX+l9VX+bRs0P/KsNhSVM4LPROSYVO7dHa8YCFEe29Fp11VWzF7/4xSkDte17dQqyygwWXF133XXTXpbu41lUyTQz6tWPGsBd3WaNfsNDYId2oA0oeV9+n3R9E5OI7aWoHezETe7nYaTDjdacBYw/aoHSyOQtuId6aDPdgOY59NBDk6zRivHcc8/NrNKiNWeBAO7mbDvykUXk7cPHs8V38sCVSX35y1+eanrfeuutIx97ri8KkkmWUPwIYHsILXkV/v/iF7+YVC+AJVrzFhCgVIhJUpMAnxT5SU/e812lVaDVGB3/ox71qGyrrbZKgcg2TzLzXVNX/j+Au+UjZbcQyg1qg/XWWy9tg/bqV786O/7441OW4re//e3M8rqqh5N7dlQs6BgB0SOOOCIt03OO0g7t7373u5PypWimAO+iNZr7HUX20Y9+NO0OTwtNLnjKKaekXZWmWSoBdUN/Ls6BOuNUSLK5+OKLmzNGHPk+Fgjgvo852vsHzfcxxxyTVCf4z5VXXjkFgXbdddfsHe94R+IX1Xf2OUtXEj389O23355efvee/0N3AGse/WGHHZa8ekFH1AxVA8C2W09sK9WO+4FklFYedYI6s6OSCV3tc3GISUykJgqAbSMQVIh7xD0oZd/9NwsSxnaMdrleBHCXs9PUP0U2iELhWUt+OfnkkxP3TSu7yiqrZIsWLUo7qwB11IraIeoeCypKgCDP2nDDDdOS9sEPfnC27LLLZssss0wqM7vFFlskHTFFAODHb1sGTwIQpm7YDnTA2Av02bmdZ2uCpaE3tlZKk5hg0XNWXxtttFF2//vfP9FoVEacgAhCTv4mCuCevM3HPqMH+Y477kjVBQUyBRE/8IEPJErF5rzSi3fYYYe0y4iHW1q62s/UCtLslV0l1Tr99NMzu9TYlQRlEtzk2EPT6AFMqLzvs88+O3ng6p6vvfbaaVI2poKYJIV15ADYuxIdcuKJJ6bJwrnEPDgA9Oaf//znQ8Pf6GjPffAA7rnt05n/Bea//OUv04ONBpFu/LnPfS55ZDzpL3zhC5mMxyuvvDIVtYqd1zsztAM7SuP/nve8J2XdWkl5qfW+yy67pAJmJmXjf+mll6YEL3kCvGabXbhPUGaSp2655ZbsxhtvTJO3+0OwEVgfcMAB6dgSwtZff/0E2LvvvnumPKvvRpuuBQK4p2v/OHtYYCwLmLBN1KpNbrnlltnqq6+eLb/88okP5yWjyQSzpdTbeenoo49OGa+UQ1ZdAt9qZFuloczosNEwK6ywQpIiWqlZzVE5RWuPBQK42zMW0ZOwwEgWQI3gmZXUFYC2qYVyBOgTQUQqIUWsBDYFFMU2lltuuRQboc+nWNl8883Tfqg87VNPPTWjVpIZKyCJopmmimUko8z4lwK4Z3yA4/IWngWAOBURL1kdHKnzF110UaLLFH+iCrETk9/FOFAkYiWC3qgTgB1A3e77JoC73eMTvQsLhAXCAn0WCODuM0m8ERYIC4QF2m2BAO52j0/0LiwQFggL9FkggLvPJPFGWCAsEBZotwUCuNs9PtG7sEBYICzQZ4EA7j6TxBthgbBAWKDdFgjgbvf4RO/CAmGBsECfBQK4+0wSb4QFwgJhgXZbIIC73eMTvQsLhAXCAn0WCODuM0m8ERYIC4QF2m2BAO52j0/0LiwQFggL9FkggLvPJPFGWCAsEBZotwUCuNs9PtG7sEBYICzQZ4EA7j6TxBthgbBAWKDdFgjgbvf4RO/CAmGBsECfBQK4+0wSb4QFwgJhgXZbIIC73eMTvQsLhAXCAn0WCODuM0m8ERYIC4QF2m2BAO52j0/0LiwQFggL9FkggLvPJPFGWCAsEBZotwUCuNs9PtG7sEBYICzQZ4EA7j6TxBthgbBAWKDdFgjgbvf4RO/CAmGBsECfBQK4+0wSb4QFwgJhgXZbIIC73eMTvQsLhAXCAn0WCODuM0m8ERYIC4QF2m2BAO52j0/0LiwQFggL9FkggLvPJPFGWCAsEBZotwUCuNs9PtG7sEBYICzQZ4EA7j6TxBthgbBAWKDdFgjgbvf4RO/CAmGBsECfBQK4+0wSb4QFwgJhgXZbIIC73eMTvQsLhAXCAn0WCODuM0m8ERYIC4QF2m2BAO52j0/0LiwQFggL9FkggLvPJPFGWCAsEBZotwUCuNs9PtG7sEBYICzQZ4EA7j6TxBthgbBAWKDdFgjgbvf4RO/CAmGBsECfBQK4+0wSb4QFwgJhgXZbIIC73eMTvQsLhAXCAn0WCODuM0m8ERYIC4QF2m2BAO52j0/0LiwQFggL9FkggLvPJPFGWCAsEBZotwUCuNs9PtG7sEBYICzQZ4EA7j6TxBthgbBAWKDdFgjgbvf4RO/CAmGBsECfBQK4+0wSb4QFwgJhgXZbIIC73eMTvQsLhAXCAn0WCODuM0m8ERYIC4QF2m2BAO52j0/0LiwQFggL9FkggLvPJPFGWCAsEBZotwUCuNs9PtG7sEBYICzQZ4EA7j6TxBthgbBAWKDdFgjgbvf4RO/CAmGBsECfBQK4+0wSb4QFwgJhgXZbIIC73eMTvQsLhAXCAn0WCODuM0m8ERYIC4QF2m2BAO52j0/0LiwQFggL9FkggLvPJPFGWCAsEBZotwUCuNs9PtG7sEBYICzQZ4EA7j6TxBthgbBAWKDdFgjgbvf4RO/CAmGBsECfBQK4+0wSb4QFwgJhgXZbIIC73eMTvQsLhAXCAn0W+P+ZPWZGBegCJwAAAABJRU5ErkJggg=="
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### softmax函数\n",
    "\n",
    "$$y_k = \\frac{exp(a_k)}{\\sum_{i=1}^{n} exp(a_i)}$$\n",
    "\n",
    "exp(x)是表示$e^x$的指数函数(e是纳皮尔常数2.7182...)。上式表示假设输出层共有 n 个神经元，计算第 k 个神经元的输出 $y_k$。\n",
    "\n",
    "用图表示 softmax 函数。\n",
    "\n",
    "![image.png](attachment:image.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 实现softmax函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.34985881, 18.17414537, 54.59815003])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 逐步实现\n",
    "import numpy as np\n",
    "\n",
    "a = np.array([0.3, 2.9, 4.0])\n",
    "\n",
    "exp_a = np.exp(a) # 指数函数\n",
    "exp_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "74.1221542101633"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum_exp_a = np.sum(exp_a) # 指数函数的和\n",
    "sum_exp_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.01821127, 0.24519181, 0.73659691])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = exp_a / sum_exp_a\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将softmax定义为函数\n",
    "def softmax(a):\n",
    "    exp_a = np.exp(a)\n",
    "    sum_exp_a = np.sum(exp_a)\n",
    "    y = exp_a / sum_exp_a\n",
    "    \n",
    "    return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$y_k = \\frac{exp(a_k)}{\\sum_{i=1}^{n} exp(a_i)} = \\frac{Cexp(a_k)}{C\\sum_{i=1}^{n} exp(a_i)}\n",
    "= \\frac{exp(a_k) + logC}{\\sum_{i=1}^{n} exp(a_i + logC)} = \\frac{exp(a_k +{C}')}{\\sum_{i=1}^{n} exp(a_i + {C}')}$$\n",
    "\n",
    "首先，在分子和分母上都乘上C这个任意的常数。然后，把这个C移动到指数函数(exp)中，记为logC。最后，把logC替换为另一个符号C'。\n",
    "\n",
    "\n",
    "这个式子说明，在进行softmax的指数函数的运算时，加上（或者减去）某个常数并不会改变运算的结果。这里的C'可以使用任何值，但是为了防止溢出，一般会使用输入信号中的最大值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E:\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:4: RuntimeWarning: overflow encountered in exp\n",
      "  after removing the cwd from sys.path.\n",
      "E:\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:4: RuntimeWarning: invalid value encountered in true_divide\n",
      "  after removing the cwd from sys.path.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([nan, nan, nan])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "a = np.array([1010, 1000, 990])\n",
    "np.exp(a) / np.sum(np.exp(a)) # softmax函数的运算\n",
    "# 数值太大，无法被正确计算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0, -10, -20])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = np.max(a) # 1010\n",
    "a - c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([9.99954600e-01, 4.53978686e-05, 2.06106005e-09])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.exp(a-c) / np.sum(np.exp(a-c))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如该例所示，通过减去输入信号中的最大值(上例中的c)，我们发现原本为 **nan(not a number，不确定)** 的地方，现在被正确计算了。综上，我们可以像下面这样实现softmax函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax(a):\n",
    "    c = np.max(a)\n",
    "    exp_a = np.exp(a-c) # 溢出对策\n",
    "    sum_exp_a = np.sum(exp_a)\n",
    "    y = exp_a / sum_exp_a\n",
    "    \n",
    "    return y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.01821127 0.24519181 0.73659691]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([0.3, 2.9, 4.0])\n",
    "y = softmax(a)\n",
    "print(y)\n",
    "np.sum(y)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
